Fine-Grained Product Class Recognition for Assisted Shopping

Assistive solutions for a better shopping experience can improve the quality of life of people, in particular also of visually impaired shoppers. We present a system that visually recognizes the fine-grained product classes of items on a shopping list, in shelves images taken with a smartphone in a grocery store. Our system consists of three components: (a) We automatically recognize useful text on product packaging, e.g., product name and brand, and build a mapping of words to product classes based on the large-scale GroceryProducts dataset. When the user populates the shopping list, we automatically infer the product class of each entered word. (b) We perform fine-grained product class recognition when the user is facing a shelf. We discover discriminative patches on product packaging to differentiate between visually similar product classes and to increase the robustness against continuous changes in product design. (c) We continuously improve the recognition accuracy through active learning. Our experiments show the robustness of the proposed method against cross-domain challenges, and the scalability to an increasing number of products with minimal re-training.Comment: Accepted at ICCV Workshop on Assistive Computer Vision and Robotics (ICCV-ACVR) 201

Multiframe visual-inertial blur estimation and removal for unmodified smartphones

Pictures and videos taken with smartphone cameras often suffer from motion blur due to handshake during the exposure time. Recovering a sharp frame from a blurry one is an ill-posed problem but in smartphone applications additional cues can aid the solution. We propose a blur removal algorithm that exploits information from subsequent camera frames and the built-in inertial sensors of an unmodified smartphone. We extend the fast non-blind uniform blur removal algorithm of Krishnan and Fergus to non-uniform blur and to multiple input frames. We estimate piecewise uniform blur kernels from the gyroscope measurements of the smartphone and we adaptively steer our multiframe deconvolution framework towards the sharpest input patches. We show in qualitative experiments that our algorithm can remove synthetic and real blur from individual frames of a degraded image sequence within a few seconds

In-air gestures around unmodified mobile devices

International audience(Communication de la commission concernant les accords d'importance mineure qui ne restreignent pas sensiblement le jeu de la concurrence au sens de l'art. 81, § 1 du traité instituant la Communauté européenne (de minimis), JOCE C. 368, 22 déc. 2001, Déc. du 25 juill. 2001, Deutsche Post - interception de courrier transfrontière, JOCE L. 331, 15 déc. 2001

m

We present a robust joint 1D and 2D barcode localization algorithm [1] on the mobile GPU. The barcode probability maps are derived from the edge and corner structures and the color of the pixels. The algorithm requires pixel-wise operations only and is hence suitable for parallel processing on mobile graphics hardware. inpu

Blur-resistant joint 1D and 2D barcode localization for smartphones

With the proliferation of built-in cameras barcode scanning on smartphones has become widespread in both consumer and enterprise domains. To avoid making the user precisely align the barcode at a dedicated position and angle in the camera image, barcode localization algorithms are necessary that quickly scan the image for possible barcode locations and pass those to the actual barcode decoder. In this paper, we present a barcode localization approach that is orientation, scale, and symbology (1D and 2D) invariant and shows better blur invariance than existing approaches while it operates in real time on a smartphone. Previous approaches focused on selected aspects such as orientation invariance and speed for 1D codes or scale invariance for 2D codes. Our combined method relies on the structure matrix and the saturation from the HSV color system. The comparison with three other real-time barcode localization algorithms shows that our approach outperforms the state of the art with respect to symbology and blur invariance at the expense of a reduced speed

Wearable machine learning for recognizing and controlling smart devices

Augmenting people with wearable technology can enhance their natural sensing, actuation, and communication capabilities. Interaction with smart devices can become easier and less explicit when combining multiple wearables instead of using device-specific apps on a single smartphone. We demonstrate a prototype for smart device control by combining quasi real-time visual device recognition on a smartphone and EMG-based gesture recognition on a Myo armband